Various forms of network-based storage systems are known today. These forms include network attached storage (NAS), storage area networks (SANs), and others. Network storage systems are commonly used for a variety of purposes, such as providing multiple users with access to shared data, backing up critical data (e.g., by data mirroring), etc.
A network-based storage system typically includes at least one storage server, which is a processing system configured to store and retrieve data on behalf of one or more client processing systems (“clients”). In the context of NAS, a storage server may be a file server, which is sometimes called a “filer”. A filer operates on behalf of one or more clients to store and manage shared files. The files may be stored in one or more arrays of mass storage devices, such as magnetic or optical disks or tapes, by using RAID (Redundant Array of Inexpensive Disks). Hence, the mass storage devices in each array may be organized into one or more separate RAID groups.
In a SAN context, a storage server provides clients with block-level access to stored data, rather than file-level access. Some storage servers are capable of providing clients with both file-level access and block-level access, such as certain Filers made by Network Appliance, Inc. (NetApp®) of Sunnyvale, Calif.
In conventional file servers, data is stored in logical containers called volumes and aggregates. An “aggregate” is a logical container for a pool of storage, combining one or more physical mass storage devices (e.g., disks) or parts thereof into a single logical storage object, which contains or provides storage for one or more other logical data sets at a higher level of abstraction (e.g., volumes). A “volume” is a set of stored data associated with a collection of mass storage devices, such as disks, which obtains its storage from (or is contained within) an aggregate, and which is managed as a single administrative unit, such as a single file system.
In conventional file servers there is a fixed, one-to-one relationship between a volume and its containing aggregate, i.e., each volume is exactly coextensive with one aggregate. Consequently, there is a fixed relationship between each volume and the disks that are associated with it. This fixed relationship means that each volume has exclusive control over the disks that are associated with the volume. Only the volume associated with the disk can read and/or write to the disk. Unused space within the disks associated with the volume cannot be used by another volume. Thus, even if a volume is only using a fraction of the space on its associated disks, the unused space is reserved for the exclusive use of the volume.
To overcome these limitations and other limitations of traditional volumes, a technology called flexible volumes has been developed by NetApp® and is now available in NetApp® Filers as a feature of the Data ONTAP™ storage operating system. A flexible volume is analogous to a traditional volume, in that it is managed as a file system, but unlike a traditional volume, it is treated separately from the underlying physical storage that contains the associated data. A “flexible volume” is, therefore, a set of stored data associated with one or more mass storage devices, such as disks, which obtains its storage from an aggregate, and which is managed as a single administrative unit, such as a single file system, but which is flexibly associated with the underlying physical storage.
Flexible volumes allow the boundaries between aggregates and volumes to be flexible, such that there does not have to be a one-to-one relationship between a flexible volume and an aggregate. An aggregate can contain multiple flexible volumes. Hence, flexible volumes can be very flexibly associated with the underlying physical storage block characteristics. Further, to help reduce the amount of wasted storage space, any free data block in an aggregate can be used by any flexible volume in the aggregate. A flexible volume can be grown or shrunk in size. Furthermore, blocks can be committed to flexible volumes on-the-fly from available storage.
While the advantages of flexible volume technology are apparent, this technology also creates certain challenges in certain areas. One such area relates to managing storage space that is reserved for special purposes, such as for storing metadata.
A volume may comprise storage in the form of discrete physical storage units such as blocks. A volume stores both user data and metadata. User data is any data generated by users of the volume that typically includes data files, application files, etc. User data is the data that is visible to a user in the volume's directory structure. Metadata is data stored by standard volumes and flexible volumes that is used to administer the user data in the volume. In general, metadata is any data stored by the volume that is not user data. Examples of metadata include tree structures describing the location of blocks corresponding to certain files in the volume, a system inode file, directory structures, etc.
Each volume includes a certain amount of reserved space (e.g., 10%) that is not reported to the user as available for storing user data. The reserved space may include space used to store metadata, as well as free space on the drives that allows more efficient operation. In one possible approach, the aggregate itself also includes reserved space, in addition to the reserves of each of the volumes stored within the aggregate. For example, if a 10% reserve is used, an aggregate totaling 1000 GB will report only 900 GB as available; the rest is reserved. Further, each time a flexible volume is created within the aggregate, 10% of the flexible volume's storage is reserved. For example, if a 100 GB flexible volume is created, only 90 GB are reported available.
Thus, in this approach, at least two reserves are created, one for the aggregate and one for each flexible volume. This approach unnecessarily duplicates at least some of the required reserve space between the aggregate and the flexible volumes within the aggregate, thereby reducing the amount of available storage available to users. What is needed, therefore, is a way to reduce the amount of reserve space used when implementing flexible volumes.